Ultrasound-based force sensing of inputs

ABSTRACT

An electronic device that senses home button inputs through ultrasonic force sensing. The electronic device may correlate that amount of force that a user applies to the home button with a specific home button command. In certain embodiments, the system may combine the force of touch information with other information that is sensed for a particular touch to correlate the touch input with a greater number of home button commands. A home button embodiment discussed herein may include a home button image that is displayed on a touch sensitive panel. In other embodiments, a home button may be located outside of the boundaries of a touch sensitive panel.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/742,500, filed Jun. 17, 2015, and entitled “Ultrasound-Based ForceSensing Inputs,” which is a continuation of U.S. patent application Ser.No. 14/417,162, filed Jan. 25, 2015, and entitled “Ultrasound-BasedForce Sensing of Inputs,” which is a 35 U.S.C. § 371 application ofPCT/US2013/032366, filed Mar. 15, 2013, and entitled “Ultrasound-BasedForce Sensing of Inputs,” and further claims the benefit under 35 U.S.C.§ 119(e) to U.S. Provisional Application No. 61/676,306, filed Jul. 26,2012, and entitled, “Ultrasound Based Force Sensing of Inputs,” all ofwhich are incorporated by reference as if fully disclosed herein.

TECHNICAL FIELD

This application generally relates to force sensing on touch devicesusing ultrasound.

BACKGROUND

Some electronic devices allow a user to enter a number of differentcommands into the electronic device through a single button, called thehome button. In these devices, pressing the home button may shift theelectronic device between various modes of operation. Typically, a homebutton is a mechanical button that recognizes two states: pressed andnot pressed. Using the home button, the device may recognize a number ofdifferent commands that correspond to the way in which a user pressesthe home button. For example, pressing the home button for a shortduration may indicate a first command, while pressing the home buttonfor a longer duration may indicate a second command. By way of furtherexample, pressing the home button in rapid succession may indicate athird command.

Because these methods of indicating commands allow only a relativelysmall number of commands to be entered, the functionality of the homebutton can be relatively limited. Accordingly, in order to expand thefunctionality of the home button, it would be advantageous todistinguish among a greater number of different ways in which a userpresses the home button. In other respects, there is an need toimplement the home button functionality without the use of a mechanicalbutton. Because some electronic devices receive other inputs through atouch screen, the use of a mechanical button adds an input mechanism ofa different type. Thus, it would be advantageous to implement the homebutton using a same or similar mechanism that is used to implement thetouch screen so that the complexity of the device is reduced. These andother needs are addressed by the following disclosure.

SUMMARY

Examples of embodiments described herein may take the form of anelectronic device that senses home button inputs through ultrasonicforce sensing. The electronic device may correlate that amount of forcethat a user applies to the home button with a specific home buttoncommand. In certain embodiments, the system may combine the force oftouch information with other information that is sensed for a particulartouch to correlate the touch input with a greater number of home buttoncommands. A home button embodiment discussed herein may include a homebutton image that is displayed on a touch sensitive panel. In otherembodiments, a home button may be located outside of the boundaries of atouch sensitive panel. One example embodiment may take the form of aforce-sensitive device having: a non-mechanical, non-depressible button;an ultrasound-based force-sensing element underlying the button withrespect to a bottom surface of the device; and a processor operative toreceive force data from the force-sensing element and interpret theforce data as a command associated with the button; wherein the commandvaries based on a force applied to the button across a continuum ofmagnitudes. Another sample embodiment may take the form of a method fordetermining and responding to an input force, comprising: ultrasonicallysensing a force exerted on an input element of an electronic device, theinput element being non-depressible and non-mechanical; determining amagnitude of the force; correlating the force to an input command, thecorrelated input command chosen from a non-binary set of possible inputcommands based on a magnitude of the force exerted on the input element;and executing the input command.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a front perspective view of a first example of a computingdevice incorporating a force sensing device;

FIG. 1B is a front perspective view of a second example of a computingdevice incorporating a force sensing device;

FIG. 1C is a front elevation view of a third example of a computingdevice incorporating the force sensing device;

FIG. 2 is a simplified cross-section view of the computing device takenalong line 2¬2 in FIG. 1A;

FIG. 3 shows a conceptual drawing of communication between a touch I/Odevice and a computing system;

FIG. 4 shows a conceptual drawing of a system including a touch sensingand force sensing I/O device;

FIG. 5 is an illustration of electronic device embodiment having a homebutton within a display screen boundary;

FIG. 6 is an illustration of electronic device embodiment having a homebutton outside of a display screen boundary;

FIG. 7A shows a conceptual drawing of a system includingultrasound-based sensing with integrated touch modules, including aliquid crystal display (LCD) construction option;

FIG. 7B shows a conceptual drawing of a system includingultrasound-based sensing with integrated touch modules, including aplastic organic light-emitting diode (OLED) construction option;

FIG. 7C shows a conceptual drawing of a system includingultrasound-based sensing with integrated touch modules, including analternative construction option;

FIG. 8A shows a conceptual drawing of a system includingultrasound-based sensing with integrated touch modules, including rowdrivers and sense columns;

FIG. 8B shows a conceptual drawing of a system includingultrasound-based sensing with integrated touch modules, includingsignals associated with row drivers and sense columns;

FIG. 9 is a flow chart that illustrates a method of processing touchinput in accordance with embodiments discussed herein;

FIG. 10 is flow chart that illustrates an example light touch processingsequence in accordance with embodiments discussed herein;

FIG. 11 is flow chart that illustrates an example medium touchprocessing sequence in accordance with embodiments discussed herein;

FIG. 12 is flow chart that illustrates an example heavy touch processingsequence in accordance with embodiments discussed herein; and

FIG. 13 is a flow chart that illustrates a method of processing touchinput in accordance with embodiments discussed herein.

FIG. 14A is a first example of a timing diagram for the computingdevice.

FIG. 14B is a second example of a timing diagram for the computingdevice.

FIG. 14C is a third example of a timing diagram for the computingdevice.

SPECIFICATION

This disclosure relates generally to an electronic device that senseshome button inputs through ultrasonic force sensing. The electronicdevice may correlate that amount of force that a user applies to thehome button with a specific home button command. In certain embodiments,the system may combine the force of touch information with otherinformation that is sensed for a particular touch to correlate the touchinput with a greater number of home button commands. A home buttonembodiment discussed herein may include a home button image that isdisplayed on a touch sensitive panel. In other embodiments, a homebutton may be located outside of the boundaries of a touch sensitivepanel.

The following terminology is exemplary, and not intended to be limitingin any way.

The text “touch sensing element”, and variants thereof, generally refersto one or more data sensing elements of any kind, including informationsensed with respect to individual locations. For example and withoutlimitation, a touch sensing element can sense data or other informationwith respect to a relatively small region of where a user is contactinga touch device.

The text “force sensing element”, and variants thereof, generally refersto one or more data sensing elements of any kind, including informationsensed with respect to force-of-touch, whether at individual locationsor otherwise. For example and without limitation, a force sensingelement can sense data or other information with respect to a relativelysmall region of where a user is forcibly contacting a device.

The text “force-of-touch”, and variants thereof, generally refers to adegree or measure of an amount of force being applied to a device. Thedegree or measure of an amount of force need not have any particularscale; for example, the measure of force-of-touch can be linear,logarithmic, or otherwise nonlinear, and can be adjusted periodically(or otherwise, such as aperiodically or otherwise from time to time) inresponse to one or more factors, either relating to force-of-touch,location of touch, time, or otherwise.

The above statements of terminology would be applicable to techniques,methods, physical elements, and systems (whether currently known orotherwise), including extensions thereof inferred or inferable by thoseskilled in the art after reading this application.

Overview

The present disclosure is generally related to a force sensing devicethat may be incorporated into a variety of electronic or computingdevices, such as, but not limited to, computers, smart phones, tabletcomputers, track pads, and so on. The force sensing device may be usedto detect one or more user force inputs on an input surface and then aprocessor (or processing element) may correlate the sensed inputs into aforce measurement and provide those inputs to the computing device. Insome embodiments, the force sensing device may be used to determineforce inputs to a track pad, a display screen, or other input surface.

The force sensing device may include an input surface, a force sensingmodule, a substrate or support layer, and optionally a sensing layerthat may detect another input characteristic than the force sensinglayer. The input surface provides an engagement surface for a user, suchas the external surface of a track pad or the cover glass for a display.In other words, the input surface may receive one or more user inputsdirectly or indirectly.

The force sensing module may include an ultrasonic module or sensor thatmay emit and detect ultrasonic pulses. In one example, the ultrasonicmodule may include a plurality of sensing elements arranged in rows orcolumns, where each of the sensing elements may selectively emit anultrasonic pulse or other signal. The pulse may be transmitted throughthe components of the force sensing device, such as through the sensinglayer and the input surface. When the pulse reaches the input surface,it may be reflected by a portion of the user (e.g., finger) or otherobject, which may reflect the pulse. The reflection of the pulse mayvary based on distance that the particular sensing element receiving thepulse is from the input. Additionally, the degree of attenuation of thepulse may also be associated with a force magnitude associated with theinput. For example, generally, as the input force on the input surfaceincreases, the contacting object exerting the force may absorb a largerpercentage of the pulse, such that the reflected pulse may be diminishedcorrespondingly.

In embodiments where it is present, the sensing layer may be configuredto sense characteristics different from the force sensing module. Forexample, the sensing layer may include capacitive sensors or othersensing elements. In a specific implantation, a multi-touch sensinglayer may be incorporated into the force sensing device and may be usedto enhance data regarding user inputs. As an example, touch inputsdetected by the sense layer may be used to further refine the forceinput location, confirm the force input location, and/or correlate theforce input to an input location. In the last example, the forcesensitive device may not use the capacitive sensing of the force sensingdevice to estimate a location, which may reduce the processing requiredfor the force sensing device. Additionally, in some embodiments, a touchsensitive device may be used to determine force inputs for a number ofdifferent touches. For example, the touch positions and force inputs maybe used to estimate the input force at each touch location.

Force Sensitive Device and System

Turning now to the figures, illustrative electronic devices that mayincorporate the force sensing device will be discussed in more detail.FIGS. 1A-1C illustrate various computing or electronic devices that mayincorporate the force sensing device. With reference to FIG. 1A, theforce sensing device may be incorporated into a computer 10, such as alaptop or desktop computer. The computer 10 may include a track pad 12or other input surface, a display 14, and an enclosure 16 or frame. Theenclosure 16 may extend around a portion of the track pad 12 and/ordisplay 14. In the embodiment illustrated in FIG. 1A, the force sensingdevice may be incorporated into the track pad 12, the display 14, orboth the track pad 12 and the display 14. In these embodiments, theforce sensing device may be configured to detect force inputs to thetrack pad 12 and/or the display 14.

In some embodiments, the force sensing device may be incorporated into atablet computer. FIG. 1B is a top perspective view of a tablet computerincluding the force sensing device. With reference to FIG. 1B, the tablecomputer 10 may include the display 14 where the force sensing device isconfigured to detect force inputs to the display 14. In addition to theforce sensing device, the display 14 may also include one or more touchsensors, such as a multi-touch capacitive grid, or the like. In theseembodiments, the display 14 may detect both force inputs, as well asposition or touch inputs.

In yet other embodiments, the force sensing device may be incorporatedinto a mobile computing device, such as a smart phone. FIG. 1C is aperspective view of a smart phone including the force sensing device.With reference to FIG. 1C, the smart phone 10 may include a display 14and a frame or enclosure 16 substantially surrounding a perimeter of thedisplay 14. In the embodiment illustrated in FIG. 1C, the force sensingdevice may be incorporated into the display 14. Similarly to theembodiment illustrated in FIG. 1B, in instances where the force sensingdevice may be incorporated into the display 14, the display 14 may alsoinclude one or more position or touch sensing devices in addition to theforce sensing device.

The force sensing device will now be discussed in more detail. FIG. 2 isa simplified cross-section view of the electronic device taken alongline 2-2 in FIG. 1A. With reference to FIG. 2, the force sensing device18 may include an input surface 20, a sensing layer 22, a force sensingmodule 24 or layer, and a substrate 28. As discussed above with respectto FIGS. 1A-1C, the input surface 20 may form an exterior surface (or asurface in communication with an exterior surface) of the track pad 12,the display 14, or other portions (such as the enclosure) of thecomputing device 10. In some embodiments, the input surface 20 may be atleast partially translucent. For example, in embodiments where the forcesensing device 18 is incorporated into a portion of the display 14.

The sensing layer 22 may be configured to sense one or more parameterscorrelated to a user input. In some embodiments, the sensing layer 22may be configured to sense characteristics or parameters that may bedifferent from the characteristics sensed by the force sensing module24. For example, the sensing layer 22 may include one or more capacitivesensors that may be configured to detect input touches, e.g.,multi-touch input surface including intersecting rows and columns. Thesensing layer 22 may be omitted where additional data regarding the userinputs may not be desired. Additionally, the sensing layer 22 mayprovide additional data that may be used to enhance data sensed by theforce sensing module 24 or may be different from the force sensingmodule. In some embodiments, there may be an air gap between the sensinglayer 22 and the force sensing module 24. In other words, the forcesensing module 24 and sensing layer may be spatially separated from eachother defining a gap or spacing distance.

The substrate 28 may be substantially any support surface, such as aportion of an printed circuit board, the enclosure 16 or frame, or thelike. Additionally, the substrate 28 may be configured to surround or atleast partially surround one more sides of the sensing device 18.

In some embodiments, a display (e.g., a liquid crystal display) may bepositioned beneath the input surface 20 or may form a portion of theinput surface 20. Alternatively, the display may be positioned betweenother layers of the force sensing device. In these embodiments, visualoutput provided by the display may be visible through the input surface20.

As generally discussed above, the force sensing device may beincorporated into one or more touch sensitive device. FIG. 3 shows aconceptual drawing of communication between a touch I/O device and acomputing system. FIG. 4 shows a conceptual drawing of a systemincluding a force sensitive touch device.

Described embodiments may include touch I/O device 1001 that can receivetouch input and force input (such as possibly including touch locationsand force of touch at those locations) for interacting with computingsystem 1003 (such as shown in FIGS. 1A-3) via wired or wirelesscommunication channel 1002. Touch I/O device 1001 may be used to provideuser input to computing system 1003 in lieu of or in combination withother input devices such as a keyboard, mouse, or possibly otherdevices. In alternative embodiments, touch I/O device 1001 may be usedin conjunction with other input devices, such as in addition to or inlieu of a mouse, trackpad, or possibly another pointing device. One ormore touch I/O devices 1001 may be used for providing user input tocomputing system 1003. Touch I/O device 1001 may be an integral part ofcomputing system 1003 (e.g., touch screen on a laptop) or may beseparate from computing system 1003.

Touch I/O device 1001 may include a touch sensitive and force sensitivepanel which is wholly or partially transparent, semitransparent,non-transparent, opaque or any combination thereof. Touch I/O device1001 may be embodied as a touch screen, touch pad, a touch screenfunctioning as a touch pad (e.g., a touch screen replacing the touchpadof a laptop), a touch screen or touchpad combined or incorporated withany other input device (e.g., a touch screen or touchpad disposed on akeyboard, disposed on a trackpad or other pointing device), anymulti-dimensional object having a touch sensitive surface for receivingtouch input, or another type of input device or input/output device.

In one example, touch I/O device 1001 embodied as a touch screen mayinclude a transparent and/or semitransparent touch sensitive and forcesensitive panel at least partially or wholly positioned over at least aportion of a display. (Although the touch sensitive and force sensitivepanel is described as at least partially or wholly positioned over atleast a portion of a display, in alternative embodiments, at least aportion of circuitry or other elements used in embodiments of the touchsensitive and force sensitive panel may be at least positioned partiallyor wholly positioned under at least a portion of a display, interleavedwith circuits used with at least a portion of a display, or otherwise.)According to this embodiment, touch I/O device 1001 functions to displaygraphical data transmitted from computing system 1003 (and/or anothersource) and also functions to receive user input. In other embodiments,touch I/O device 1001 may be embodied as an integrated touch screenwhere touch sensitive and force sensitive components/devices areintegral with display components/devices. In still other embodiments atouch screen may be used as a supplemental or additional display screenfor displaying supplemental or the same graphical data as a primarydisplay and to receive touch input, including possibly touch locationsand force of touch at those locations.

Touch I/O device 1001 may be configured to detect the location of one ormore touches or near touches on device 1001, and where applicable, forceof those touches, based on capacitive, resistive, optical, acoustic,inductive, mechanical, chemical, or electromagnetic measurements, inlieu of or in combination or conjunction with any phenomena that can bemeasured with respect to the occurrences of the one or more touches ornear touches, and where applicable, force of those touches, in proximityto device 1001. Software, hardware, firmware or any combination thereofmay be used to process the measurements of the detected touches, andwhere applicable, force of those touches, to identify and track one ormore gestures. A gesture may correspond to stationary or non-stationary,single or multiple, touches or near touches, and where applicable, forceof those touches, on touch I/O device 1001. A gesture may be performedby moving one or more fingers or other objects in a particular manner ontouch I/O device 1001 such as tapping, pressing, rocking, scrubbing,twisting, changing orientation, pressing with varying pressure and thelike at essentially the same time, contiguously, consecutively, orotherwise. A gesture may be characterized by, but is not limited to apinching, sliding, swiping, rotating, flexing, dragging, tapping,pushing and/or releasing, or other motion between or with any otherfinger or fingers, or any other portion of the body or other object. Asingle gesture may be performed with one or more hands, or any otherportion of the body or other object by one or more users, or anycombination thereof.

Computing system 1003 may drive a display with graphical data to displaya graphical user interface (GUI). The GUI may be configured to receivetouch input, and where applicable, force of that touch input, via touchI/O device 1001. Embodied as a touch screen, touch I/O device 1001 maydisplay the GUI. Alternatively, the GUI may be displayed on a displayseparate from touch I/O device 1001. The GUI may include graphicalelements displayed at particular locations within the interface.Graphical elements may include but are not limited to a variety ofdisplayed virtual input devices including virtual scroll wheels, avirtual keyboard, virtual knobs or dials, virtual buttons, virtuallevers, any virtual UI, and the like. A user may perform gestures at oneor more particular locations on touch I/O device 1001 which may beassociated with the graphical elements of the GUI. In other embodiments,the user may perform gestures at one or more locations that areindependent of the locations of graphical elements of the GUI. Gesturesperformed on touch I/O device 1001 may directly or indirectlymanipulate, control, modify, move, actuate, initiate or generally affectgraphical elements such as cursors, icons, media files, lists, text, allor portions of images, or the like within the GUI. For instance, in thecase of a touch screen, a user may directly interact with a graphicalelement by performing a gesture over the graphical element on the touchscreen. Alternatively, a touch pad generally provides indirectinteraction. Gestures may also affect non-displayed GUI elements (e.g.,causing user interfaces to appear) or may affect other actions withincomputing system 1003 (e.g., affect a state or mode of a GUI,application, or operating system). Gestures may or may not be performedon touch I/O device 1001 in conjunction with a displayed cursor. Forinstance, in the case in which gestures are performed on a touchpad, acursor (or pointer) may be displayed on a display screen or touch screenand the cursor may be controlled via touch input, and where applicable,force of that touch input, on the touchpad to interact with graphicalobjects on the display screen. In other embodiments in which gesturesare performed directly on a touch screen, a user may interact directlywith objects on the touch screen, with or without a cursor or pointerbeing displayed on the touch screen.

Feedback may be provided to the user via communication channel 1002 inresponse to or based on the touch or near touches, and where applicable,force of those touches, on touch I/O device 1001. Feedback may betransmitted optically, mechanically, electrically, olfactory,acoustically, haptically, or the like or any combination thereof and ina variable or non-variable manner.

Attention is now directed towards embodiments of a system architecturethat may be embodied within any portable or non-portable deviceincluding but not limited to a communication device (e.g. mobile phone,smart phone), a multi-media device (e.g., MP3 player, TV, radio), aportable or handheld computer (e.g., tablet, netbook, laptop), a desktopcomputer, an All-In-One desktop, a peripheral device, or any other(portable or non-portable) system or device adaptable to the inclusionof system architecture 2000, including combinations of two or more ofthese types of devices. FIG. 4 is a block diagram of one embodiment ofsystem 2000 that generally includes one or more computer-readablemediums 2001, processing system 2004, Input/Output (I/O) subsystem 2006,electromagnetic frequency (EMF) circuitry (such as possibly radiofrequency or other frequency circuitry) 2008 and audio circuitry 2010.These components may be coupled by one or more communication buses orsignal lines 2003. Each such bus or signal line may be denoted in theform 2003-X, where X can be a unique number. The bus or signal line maycarry data of the appropriate type between components; each bus orsignal line may differ from other buses/lines, but may perform generallysimilar operations.

It should be apparent that the architecture shown in FIGS. 1A-4 are onlyone example architecture of system 2000, and that system 2000 could havemore or fewer components than shown, or a different configuration ofcomponents. The various components shown in FIGS. 3-4 can be implementedin hardware, software, firmware or any combination thereof, includingone or more signal processing and/or application specific integratedcircuits.

EMF circuitry 2008 is used to send and receive information over awireless link or network to one or more other devices and includeswell-known circuitry for performing this function. EMF circuitry 2008and audio circuitry 2010 are coupled to processing system 2004 viaperipherals interface 2016. Interface 2016 includes various knowncomponents for establishing and maintaining communication betweenperipherals and processing system 2004. Audio circuitry 2010 is coupledto audio speaker 2050 and microphone 2052 and includes known circuitryfor processing voice signals received from interface 2016 to enable auser to communicate in real-time with other users. In some embodiments,audio circuitry 2010 includes a headphone jack (not shown).

Peripherals interface 2016 couples the input and output peripherals ofthe system to processor 2018 and computer-readable medium 2001. One ormore processors 2018 communicate with one or more computer-readablemediums 2001 via controller 2020. Computer-readable medium 2001 can beany device or medium that can store code and/or data for use by one ormore processors 2018. Medium 2001 can include a memory hierarchy,including but not limited to cache, main memory and secondary memory.The memory hierarchy can be implemented using any combination of RAM(e.g., SRAM, DRAM, DDRAM), ROM, FLASH, magnetic and/or optical storagedevices, such as disk drives, magnetic tape, CDs (compact disks) andDVDs (digital video discs). Medium 2001 may also include a transmissionmedium for carrying information-bearing signals indicative of computerinstructions or data (with or without a carrier wave upon which thesignals are modulated). For example, the transmission medium may includea communications network, including but not limited to the Internet(also referred to as the World Wide Web), intranet(s), Local AreaNetworks (LANs), Wide Local Area Networks (WLANs), Storage Area Networks(SANs), Metropolitan Area Networks (MAN) and the like.

One or more processors 2018 run various software components stored inmedium 2001 to perform various functions for system 2000. In someembodiments, the software components include operating system 2022,communication module (or set of instructions) 2024, touch andforce-of-touch processing module (or set of instructions) 2026, graphicsmodule (or set of instructions) 2028, one or more applications (or setof instructions) 2030, and fingerprint sensing module (or set ofinstructions) 2038. Each of these modules and above noted applicationscorrespond to a set of instructions for performing one or more functionsdescribed above and the methods described in this application (e.g., thecomputer-implemented methods and other information processing methodsdescribed herein). These modules (i.e., sets of instructions) need notbe implemented as separate software programs, procedures or modules, andthus various subsets of these modules may be combined or otherwiseRe-arranged in various embodiments. In some embodiments, medium 2001 maystore a subset of the modules and data structures identified above.Furthermore, medium 2001 may store additional modules and datastructures not described above.

Operating system 2022 includes various procedures, sets of instructions,software components and/or drivers for controlling and managing generalsystem tasks (e.g., memory management, storage device control, powermanagement, etc.) and facilitates communication between various hardwareand software components.

Communication module 2024 facilitates communication with other devicesover one or more external ports 2036 or via EMF circuitry 2008 andincludes various software components for handling data received from EMFcircuitry 2008 and/or external port 2036.

Graphics module 2028 includes various known software components forrendering, animating and displaying graphical objects on a displaysurface. In embodiments in which touch I/O device is a touch sensitiveand force sensitive display (e.g., touch screen), graphics module 2028includes components for rendering, displaying, and animating objects onthe touch sensitive and force sensitive display.

One or more applications 2030 can include any applications installed onsystem 2000, including without limitation, a browser, address book,contact list, email, instant messaging, word processing, keyboardemulation, widgets, JAVA-enabled applications, encryption, digitalrights management, voice recognition, voice replication, locationdetermination capability (such as that provided by the globalpositioning system, also sometimes referred to herein as “GPS”), a musicplayer, and otherwise.

In one embodiment, the system may implement a “home screen” that isdisplayed so that a user may select from a number of applicationsinstalled on the system. A home screen embodiment may include an arrayof selectable icons arranged in rows and columns and displayed on adisplay screen. Each icon may represent a particular application andwhen the user selects the icon by touch or otherwise, the systemlaunches the application. The icons may represent applications that havebeen added to the system by the user or applications that are built intothe system. One example of an application that is built into the systemis a music player.

As used herein, when the system displays the home screen or runs anapplication, the system is said to be in a specific “mode of operation.”Examples of other modes of operation include a recently-used applicationselector interface, a search function interface, a lock screen and avoice command interface. As described in greater detail below, thesystem may implement a “home button” that allows the user to shift thesystem between various modes of operation.

A “recently-used application selector interface” may be a menu thatcontains an icon for each application that was recently launched by theuser. In one embodiment, the system displays the recently-usedapplication selector interface in the bottom portion of the displayscreen. In this or other embodiments, the recently-used applicationselector interface displays icons for the four most recently launchedapplications.

A “search function interface” may be displayed to allow a user to searchfor content that is stored on the system. In one embodiment, the searchfunction interface includes a dialog box and an input mechanism such asvirtual keypad or keyboard by which the user may enter a search string.Once the user has entered a search string, the search function interfacemay search the device to find related content.

A “lock screen” may be displayed to indicate that system is locked andtherefore not accessible without password entry. In one embodiment, thelock screen displays a numeric key pad that allows a user to enter theappropriate password. In this or other embodiments, the system maydisable itself after a predetermined number of failed password entryattempts.

A “voice command interface” may be displayed to allow a user to entercertain commands by speaking into the device. In one embodiment, thevoice command interface includes a virtual microphone or otherappropriate icon that is displayed on a portion of the display surface.When the user speaks a command into the device, the voice commandinterface may execute the command, as appropriate.

Touch and force-of-touch processing module 2026 includes varioussoftware components for performing various tasks associated with touchI/O device including but not limited to receiving and processing touchinput and force-of-touch input received from I/O device 2012 via touchI/O device controller 2032.

System 2000 may further include fingerprint sensing module 2038 forperforming the method/functions as described herein in connection withother figures shown and described herein.

I/O subsystem 2006 is coupled to touch I/O device and one or more otherI/O devices 2014 for controlling or performing various functions. TouchI/O device communicates with processing system 2004 via touch I/O devicecontroller 2032, which includes various components for processing usertouch input and force-of-touch input (e.g., scanning hardware). One ormore other input controllers 2034 receives/sends electrical signalsfrom/to other I/O devices 2014. Other I/O devices 2014 may includephysical buttons, dials, slider switches, sticks, keyboards, touch pads,additional display screens, or any combination thereof.

If embodied as a touch screen, touch I/O device displays visual outputto the user in a GUI. The visual output may include text, graphics,video, and any combination thereof. Some or all of the visual output maycorrespond to user-interface objects. Touch I/O device forms atouch-sensitive and force-sensitive surface that accepts touch input andforce-of-touch input from the user. Touch I/O device and touch screencontroller 2032 (along with any associated modules and/or sets ofinstructions in medium 2001) detects and tracks touches or near touches,and where applicable, force of those touches (and any movement orrelease of the touch, and any change in the force of the touch) on touchI/O device and converts the detected touch input and force-of-touchinput into interaction with graphical objects, such as one or moreuser-interface objects. In the case in which device 2012 is embodied asa touch screen, the user can directly interact with graphical objectsthat are displayed on the touch screen. Alternatively, in the case inwhich device 2012 is embodied as a touch device other than a touchscreen (e.g., a touch pad or trackpad), the user may indirectly interactwith graphical objects that are displayed on a separate display screenembodied as I/O device 2014.

Touch I/O device may be analogous to the multi-touch sensitive surfacedescribed in the following U.S. Pat. Nos. 6,323,846, 6,570,557, and/or6,677,932, and/or U.S. Patent Publication 2002/0015024A1, each of whichis hereby incorporated by reference.

Embodiments in which touch I/O device is a touch screen, the touchscreen may use LCD (liquid crystal display) technology, LPD (lightemitting polymer display) technology, OLED (organic LED), or OEL(organic electro luminescence), although other display technologies maybe used in other embodiments.

Feedback may be provided by touch I/O device based on the user's touch,and force-of-touch, input as well as a state or states of what is beingdisplayed and/or of the computing system. Feedback may be transmittedoptically (e.g., light signal or displayed image), mechanically (e.g.,haptic feedback, touch feedback, force feedback, or the like),electrically (e.g., electrical stimulation), olfactory, acoustically(e.g., beep or the like), or the like or any combination thereof and ina variable or non-variable manner.

System 2000 also includes power system 2044 for powering the varioushardware components and may include a power management system, one ormore power sources, a recharging system, a power failure detectioncircuit, a power converter or inverter, a power status indicator and anyother components typically associated with the generation, managementand distribution of power in portable devices.

In some embodiments, peripherals interface 2016, one or more processors2018, and memory controller 2020 may be implemented on a single chip,such as processing system 2004. In some other embodiments, they may beimplemented on separate chips.

FIG. 5 is an illustration of an electronic device embodiment 3000 inaccordance with embodiments discussed herein. The electronic device 3000is adapted to allow a user to enter a number of different commands intothe electronic device 3000 through a single button, called the homebutton. The electronic device 3000 is configured to distinguish betweenor among the different commands entered through the home button by theamount of force by which the user presses the home button. In oneembodiment, a light touch may indicate a first command and a heaviertouch may indicate a second command. In another embodiment, a lighttouch may indicate a first command, a medium touch may indicate a secondcommand, and a heavier touch may indicate a third command.

By way of example, when the electronic device 3000 is running anapplication, the electronic device 3000 may be configured to receive afirst home-button command that causes the device to exit theapplication. Further, the electronic device 3000 may also be configuredto receive a second home-button command that causes the device to entera recently-used application selector interface. The electronic device3000 may distinguish between the first and second command by measuringthe amount of force the user applies when she presses the home button.Here, a light touch may indicate the application exit command and amedium touch may indicate the command to enter the application selectorinterface. In another example, the electronic device may distinguishbetween these two commands, as well as a third command indicated by aheavy touch.

The set of different commands that may be entered through the homebutton may vary depending on the mode in which the electronic device3000 is running, For example, as described above, when the electronicdevice 3000 is running an application, the electronic device 3000 may beconfigured to receive a first home button command that causes the deviceto exit the application and a second home button command that causes thedevice to enter a recently-used application selector interface. When theelectronic device 3000 is not running an application, the electronicdevice 3000 may be configured to receive a different set of commandsthrough the operation of the home button. For example, if the device iscurrently at the home screen, the electronic device 3000 may beconfigured to receive a home-button command to enable a search functioninterface. If the device is at the lock screen, the electronic device3000 may be configured to receive a home-button command to enable amusic player and to display a panel of music player controls.

In some embodiments, the electronic device may be configured to receivethe same home button command in more than one mode of operation. Morespecifically, a particular level of force may correspond to the samehome button command across various modes of operation for the electronicdevice. For example, a home button command that enables the voicecommand interface may be available both when the electronic device isrunning an application and when the electronic device is at the homescreen. In one embodiment, pressing the home button with a heavy touchwhile the device is running an application may cause the electronicdevice to execute the voice command interface. Similarly, pressing thehome button with a heavy touch while the device is at the home screenmay cause the electronic device to execute the voice command interface.Pressing the home button with a heavy touch may also cause theelectronic device to execute the voice command interface in other modesof operation. For example, pressing the home button with a heavy touchwhile the device is at the lock screen may also cause the electronicdevice to execute the voice command interface.

Throughout this disclosure, the operation of the home button isdescribed as executing certain “home-button commands.” Typically, a“home-button command” is a higher level command than an applicationlevel command that may be entered through other input mechanisms, suchas touch input on a portion of a touch screen that does not include thehome-button. For example, a home-button command may be one that switchesthe mode of operation of the device. As describe above, home buttoncommand embodiments include those that cause the electronic device toexit a currently running application and enter the home screen. Incontrast, an application level command may be one that is input throughother input mechanisms and may be directed a particular application thatis currently running on the electronic device. More specifically,application level commands may correlated with icons or other graphicsthat are displayed for a temporary period of time. In contrast to theseicons or other graphics, the home button typically persists over timeand is available for use anytime the device is in operation. In mostembodiments, the home button is always found in the same location on theelectronic device. Although a “home-button command” is typicallydescribed herein as a higher level command there is no particularrequirement for this. Accordingly, a device consistent with thisdisclosure could implement an application level command that thisentered through the home button.

FIG. 5 is an illustration of an electronic device embodiment 3000 thatcontains a home button 3005 that is within the boundaries of a touchsensitive display panel 3010. In some embodiments, the touch sensitivedisplay panel 3010 is located within a black mask 3015 or other borderarea 3015 that contains other elements such as a speaker 3020 and acamera 3025 The electronic device 3000 embodiment shown in FIG. 3implements the home button 3005 as an image that is displayed by animage-producing element that underlies the touch sensitive display panel3010. In addition to the image-producing element, a force sensingelement also underlies the touch sensitive display panel 3010 directlybeneath the home button image in order to measure the force with which auser presses the home button 3005 image. The structure that underliesthe home button 3005 shown in FIG. 5 is described in greater detail inconnection with FIG. 7A and FIG. 7B.

FIG. 6 is an illustration of an electronic device embodiment 4000 thatcontains a home button 4005 that is outside the boundaries of a touchsensitive display panel 4010, within the black mask area 3015. Morespecifically, The home button 4005 may be located within a border area3015 that contains other elements such as a speaker 3020 and a camera3025 The electronic device 4000 may include an icon or other image onthe glass that indicates the location of the home button. In variousembodiments, the icon may be a printed or stamped image; or the icon maybe cut or etched into the glass.

The home button 4005 may function the same as the home button 3005 thatis shown in FIG. 5. Nevertheless, the electronic device 4000 mayimplement the home button 4005 without the use an image-producingelement that underlies the touch sensitive display panel 3010. Like theelectronic device embodiment shown in FIG. 5, a force sensing elementunderlies the touch sensitive display panel 3010 directly beneath thehome button in order to measure the force with which a user presses thehome button 3005. The structure that underlies the home button 4005shown in FIG. 4 is described in greater detail in connection with FIG.7C.

FIG. 7A shows a conceptual drawing of a system includingultrasound-based sensing, including a liquid crystal display (LCD)construction option. FIG. 7A shows a system 3030 includingultrasound-based sensing with includes a touch I/O device 3000 asdescribed herein, including a cover glass (CG) element, which may betouched by the user, and for which force-of-touch may be sensed. Anultrasound-based force sensing element is disposed below the coverglass.

In one embodiment, the touch I/O device can include a liquid crystaldisplay (LCD) construction option with a cover glass (CG) element 302that, in some implementations, may have a thickness of approximately 600microns. The cover glass (CG) element 302 might be used to receive touchand force of touch from the user. The cover glass (CG) element 302 canbe constructed using one or more layers of glass, chemically treatedglass, sapphire, or one or more other substances.

In one embodiment, the liquid crystal display (LCD) construction optioncan include a first optically clear adhesive (OCA) element 304 disposedbelow the cover glass element, which in some implementations, can have athickness of approximately 150 microns.

In one embodiment, the liquid crystal display (LCD) construction optioncan include a top point of load (POL) element 306 disposed below thefirst optically clear adhesive (OCA) element 304, which, in someimplementations, can have a thickness of approximately 200 microns. Thetop POL element 306 might be used to distribute power to elements of thetouch I/O device.

In one embodiment, the liquid crystal display (LCD) construction optioncan include a CF glass element 308 disposed below the top POL element306, which, in some implementations, can have a thickness ofapproximately 150 microns.

In one embodiment, the liquid crystal display (LCD) construction optioncan include a thin film transistor (TFT) LCD glass element 310 disposedbelow the top POL element 306, which, in some implementations, can havea thickness of approximately 150 microns. The TFT LCD element 310 mightbe used to present display elements for the touch I/O device. Asdescribed above in connection with FIG. 5, the TFT LCD element may beused to display a home-button image.

In one embodiment, the liquid crystal display (LCD) construction optioncan include a bottom POL element 312 disposed below the TFT LCD glasselement 310, which, in some implementations, can have a thickness ofapproximately 200 microns. Similar to the top POL element 306, thebottom POL element 312 might be used to distribute power to elements ofthe touch device 2012.

In one embodiment, the liquid crystal display (LCD) construction optioncan include a second optically clear adhesive (OCA) element 314 disposedbelow the bottom POL element 312, which, in some implementations, canhave a thickness of approximately 150 microns.

In one embodiment, the liquid crystal display (LCD) construction optioncan include an ultrasonic sensing element 316, which may bepolyvinylidene difluoride (PVDF), disposed below the second opticallyclear adhesive (OCA) element 314, which, in some implementations, canhave a thickness of approximately 50 microns.

In one embodiment, the liquid crystal display (LCD) construction optioncan include a set of backlight layers 318 disposed below the ultrasonicsensing element 316. The backlight layers 318, in combination with theTFT LCD glass element, can provide the touch I/O device with a displaycapability.

In one embodiment, the liquid crystal display (LCD) construction optioncan include a set of semi-transparent sense column circuits 320,disposed below the second optically clear adhesive (OCA) element 314.For example, the semi-transparent sense column circuits 320 can includeone or more metal layers or metallized layers, in which circuit elementsare disposed. In one such example, the circuit elements are disposed ina set of horizontal and vertical wire elements, located so as not tocover the TFT LCD elements (or other related display elements), with theeffect that the circuits do not cause the lighted elements of thedisplay to be substantially hindered or obscured. Similarly, in oneembodiment, the liquid crystal display (LCD) construction option caninclude a second of semi-transparent row driver circuits 322, disposedbelow the ultrasonic sensing element 316. For example, thesemi-transparent row driver circuits 322 can also include one or moremetal layers or metallized layers, in which circuit elements aredisposed. In one such example, the circuit elements are disposed in aset of horizontal and vertical wire elements, located so as not to coverthe TFT LCD elements (or other related display elements), with theeffect that the circuits do not cause the lighted elements of thedisplay to be substantially hindered or obscured. The sense columncircuits and the row driver circuits are further described below.

FIG. 7B shows a conceptual drawing of a system includingultrasound-based sensing, including a plastic organic light-emittingdiode (OLED) construction option. FIG. 7B shows a system includingultrasound-based sensing, including a cover glass (CG) element 402,which may be touched by the user, and for which force-of-touch may besensed. An ultrasound-based force sensing element is disposed below thecover glass.

In one embodiment, the touch I/O device 4000 can include a plasticorganic light-emitting diode (OLED) construction option. The plasticorganic light-emitting diode (OLED) construction option can include thecover glass (CG) element 402, which, in some implementations, can have athickness of approximately 600 microns. The cover glass (CG) element 402might be used to receive touch and force of touch from the user. Thecover glass (CG) element 402 can be constructed using one or more layersof glass, chemically treated glass, sapphire, or one or more othersubstances.

In one embodiment, the plastic organic light-emitting diode (OLED)construction option can include a first optically clear adhesive (OCA)element 404 disposed below the cover glass element, which, in someimplementations, can have a thickness of approximately 150 microns.

In one embodiment, the plastic organic light-emitting diode (OLED)construction option can include a plastic film dual indium-titaniumoxide (DITO) element 406 disposed below the first optically clearadhesive (OCA) element 404, which, in some implementations, may havethickness of approximately 115 microns.

In one embodiment, the plastic organic light-emitting diode (OLED)construction option can include a second optically clear adhesive (OCA)element 408 disposed below the DITO element 406, which, in someimplementations, can have a thickness of approximately 150 microns.

In one embodiment, the plastic organic light-emitting diode (OLED)construction option can include a plastic OLED display element 410disposed below the second optically clear adhesive (OCA) element 408,which, in some implementations, can have a thickness of approximately250 microns. As described above in connection with FIG. 6, the plasticOLED element may be used to display a home-button image.

In one embodiment, the plastic organic light-emitting diode (OLED)construction option can include an ultrasonic sensing element 414disposed below the plastic OLED display element 410, which, in someimplementations, can have a thickness of approximately 50 microns.

In one embodiment, the plastic organic light-emitting diode (OLED)construction option can include a foam or rubber element 418 disposedbelow the ultrasonic sensing element 414, and a mid-plate element 420(such as a support element) disposed below the foam or rubber element418.

In one embodiment, the plastic organic light-emitting diode (OLED)construction option can include a set of semi-transparent sense columncircuits 412, disposed below the second optically clear adhesive (OCA)element 408. For example, the semi-transparent sense column circuits 412can include one or more metal layers or metallized layers, in whichcircuit elements are disposed. In one such example, the circuit elementsare disposed in a set of horizontal and vertical wire elements, locatedso as not to cover the plastic OLED elements (or other related displayelements), with the effect that the circuits do not cause the lightedelements of the display to be substantially hindered or obscured.Similarly, in one embodiment, the plastic organic light-emitting diode(OLED) construction option can include a second of semi-transparent rowdriver circuits 416, disposed below the ultrasonic sensing element 414.For example, the semi-transparent row driver circuits 416 can includeone or more metal layers or metallized layers, in which circuit elementsare disposed. In one such example, the circuit elements are disposed ina set of horizontal and vertical wire elements, located so as not tocover the plastic OLED elements (or other related display elements),with the effect that the circuits do not cause the lighted elements ofthe display to be substantially hindered or obscured. The sense columncircuits and the row driver circuits are further described below.

FIG. 7C is an illustration of construction option embodiment thatunderlies the home button 4005 without the use an image-producingelement that underlies the touch sensitive display panel 3010. Theelectronic device 400 does, however, include a force sensing elementunderlying the touch sensitive display panel 3010 directly beneath thehome button in order to measure the force with which a user presses thehome button 3005. In this regard, the construction option embodimentshown in FIG. 7C can include a set of semi-transparent sense columncircuits, disposed below the second optically clear adhesive (OCA)element. For example, the semi-transparent sense column circuits caninclude one or more metal layers or metallized layers, in which circuitelements are disposed. In one such example, the circuit elements aredisposed in a set of horizontal and vertical wire elements. Similarly,in one embodiment, the construction option embodiment shown in FIG. 7Ccan include a second of semi-transparent row driver circuits, disposedbelow the ultrasonic sensing element. For example, the semi-transparentrow driver circuits can include one or more metal layers or metallizedlayers, in which circuit elements are disposed. In one such example, thecircuit elements are disposed in a set of horizontal and vertical wireelements. The sense column circuits and the row driver circuits arefurther described below.

FIG. 8A shows a conceptual drawing of a system includingultrasound-based sensing with integrated touch modules, including rowdrivers and sense columns.

FIG. 8B shows a conceptual drawing of a system includingultrasound-based sensing, including signals associated with row driversand sense columns.

In one embodiment, the ultrasound-based sensing element includes one ormore rows and one or more columns, disposed in an overlapping manner,such as rectilinearly, with the effect of identifying one or more forcesensing elements at each intersection of a particular such row and aparticular such column. This has the effect that force of touch can bedetermined independently at each particular one such force sensingelement.

In one embodiment, the ultrasound-based sensing elements have their rowscoupled to one or more triggering and driving circuits (such as shown inthe figure as TX1 and TX2, corresponding to rows 1 and 2, respectively),each of which is coupled to a corresponding row of the ultrasound-basedsensing element. Each corresponding row of the ultrasound-based sensingelement is coupled to a sequence of one or more ultrasound-basedsensors. Each ultrasound-based sensor can, when triggered, emit anultrasonic pulse or other signal (such as shown in the figure as TX1 andTX1, again corresponding to rows 1 and 2, respectively), which istransmitted from the ultrasound-based sensor, through the elementsdescribed with respect to the FIG. 5A or the FIG. 5B, and to the surfaceof the cover glass.

The triggering and driving circuits generate one or more pulses whichare transmitted to the rows of the ultrasound-based sensing device, eachof which is coupled to a corresponding row of individualultrasound-based sensing elements. Similarly, in one embodiment, theindividual ultrasound-based sensing elements have their columns coupledto one or more sensing and receiving circuits, each of which is coupledto a corresponding column of the ultrasound-based sensing device.Collectively, this has the effect that one or more rows of theultrasound-based sensing device are driven by corresponding triggeringsignals, which are coupled to one or more columns of theultrasound-based sensing device, which are sensed by correspondingreceiving circuits.

When the ultrasonic pulse of the ultrasound-based sensing device reachesthe front surface of the cover glass, it would be reflected, at least inpart, by at least the glass-to-air interface. With the user's fingertip,or other part of the user's body, or other touching element (such as asoft-ended stylus or similar device) in contact with the glass, theamount of the ultrasonic pulse that is reflected varies with the amountof force that is applied to the glass. Specifically, the user'sfingertip absorbs a certain amount of the energy from the ultrasonicpulse thus reducing the energy in the reflected pulse. Further, theamount of energy absorbed by the fingertip increases as the fingertipapplies a greater amount of force to the glass. Thus, as the fingertipapplies a greater amount of force to the glass, a lesser amount ofenergy is reflected back from the glass-to-air interface. When theultrasonic pulse that is reflected back to the ultrasound-based sensorwhich emitted that ultrasonic pulse, the amplitude and thus the energyof the reflected pulse can be measured. Further the amplitude of thereflected pulse can be then correlated with different amounts of force,as applied to the glass.

In one embodiment, the system can define two levels of force, such aslight and medium, that are correlated with two ranges of energy in thereflected pulse. In other embodiments, the system can define threelevels of force, such as light, medium and heavy, that are correlatedwith the three ranges of energy in the reflected pulse. It should beappreciated that these correlations are by way of example and notlimitation. In accordance with other embodiments, any number of forcelevels may be defined as is appropriate for a particular design.

One or more such reflections from the front surface of the cover glasscan be identified by the columns of the ultrasound-based sensing element(such as shown in the figure as Vout A, Vout B, and Vout C,corresponding to columns A, B, and C, respectively). Each such column iscoupled to a sense amplifier, such as shown in the figure including areference voltage Vref (such as a grounding voltage or other referencevoltage), an amplifier, and a feedback impedance element (such as acapacitor, resistor, or combination or conjunction thereof, orotherwise). Although each sense amplifier is shown in the figure ascoupled to only one sensing element, in the context of the invention,there is no particular requirement for any such limitation. For example,one or more such sense amplifiers can include a differential senseamplifier, or other sense amplifier design.

In one embodiment, each sense amplifier is disposed so that it generatesa relatively maximal response in those cases when the ultrasonicreflection from the front of the cover glass is due to a force directlyabove the force sense element. This has the effect that when the forcesense element receives a force of touch from the user, the relativelymaximal response to that force of touch impressed on the cover glass bythe user is primarily from the ultrasound-based sensing element at theindividual row/column associated with the location where that force oftouch is relatively maximal. To the extent that force of touch impressedon the cover glass by the user is also impressed on other locations onthe cover glass, the ultrasound-based sensing element at the individualrow/column associated with those other locations would also beresponsive.

In one embodiment, each sense amplifier is also disposed so that itgenerates a relatively minimal response in those cases when theultrasonic reflection from the front of the cover glass is due to aforce from a location relatively far from directly above the force senseelement. For example, in the case that the ultrasonic reflection is froma portion of the ultrasonic pulse which radiates at an angle from theultrasound-based sensor, and is similarly reflected back at that angle,the arrival time of that ultrasonic pulse would be sufficientlydifferent from a direct up-and-down reflection that the sense amplifiercan be disposed to disregard that portion of the reflection of theultrasonic pulse. This has the effect that the sense amplifier can bedisposed to only respond to those cases when force of touch is impressedon the cover glass by the user directly above the sense amplifier.

For example, an ultrasonic pulse can be generated by a triggering pulsefrom a driving circuit, such as TX1 or TX2, with the effect of providinga first set of (unwanted) reflections and a second set of (wanted)reflections, one set for each of Vout A, Vout B, and Vout C. Theunwanted reflections might be responsive to reflections from otherultrasonic pulses, from ultrasonic pulses that are reflected fromelements other than the front of the cover glass, or otherwise. Forexample, the unwanted reflections might occur at a time after thetriggering pulse from driving circuit, such as less than about 450nanoseconds after the triggering pulse, but before an expected time forthe ultrasonic pulse to travel to the front of the cover glass and bereflected, such as more than about 450 nanoseconds after the triggeringpulse. In such cases, the receiving and sensing circuits would bedisposed to decline to respond to those reflections which are not withinthe expected window of time duration for a response from the correctforce sensing element.

In one embodiment, the touch I/O device can include a capacitive touchsensing device, which can determine a location, or an approximatelocation, at which the user contacts, or nearly contacts, the touch I/Odevice, such as in combination with the ultrasound-based sensing device.For example, the capacitive touch sensing device can include a set ofcapacitive touch sensors, each of which is disposed to determine if theuser contacts, or nearly contacts, the touch I/O device at one or morecapacitive touch sensing elements, such as in response to an row-driverelectronic pulse as can be used to also drive the ultrasound-basedsensing device. In such cases, the capacitive touch sensing deviceincludes a substantially quicker signal response than theultrasound-based sensing device, and this substantially quicker signalresponse can be detected by the capacitive touch sensing device outsidea time window otherwise used by the ultrasound-based sensing device.

In one embodiment, the triggering pulse from a driving circuit, such asTX1 or TX2, provides a set of triggers for touch sensing elements andforce sensing elements in that row. For example, a triggering pulse TX1can provide a set of triggers in a first row for both capacitive touchsensing devices and ultrasound-based sensing devices, the formerproviding an activated capacitive element (such as an activatedcapacitive plate) and the latter providing an activated ultrasound-basedsensing element (such as an activated ultrasonic pulse). In such cases,in response to a contact, or near contact, the capacitive touch sensingdevice would be activated relatively quickly, in general, substantiallymore quickly than an ultrasonic reflection would be received.

In such cases, this would have the effect that responsive signalsmeasured at Vout A, Vout B, and Vout C would have multiple components:(A) a first component including a relatively quick response to theactivated capacitive element, (B) a second component including a set ofearlier, and relatively unwanted, responses to the ultrasound-basedsensing device, as described above which might occur less than 450nanoseconds after the ultrasonic pulse, and (C) a third componentincluding a set of later, and relatively wanted, responses to theultrasound-based sensing device, as described above which might occurmore than 450 nanoseconds after the ultrasonic pulse. Each of thesecomponents can be separately filtered, with the first component ofrelatively quick responses being routed to the capacitive sensingdevice, the second component of relatively unwanted responses beingdiscarded, and the third component of relatively wanted responses beingrouted to the ultrasound-based sensing device.

In one embodiment, the touch I/O device can combine information from thecapacitive touch sensing device and the ultrasound-based force sensingdevice, with the effect of determining both a location of touch and aforce of touch by the user.

In one embodiment, the touch I/O device can maintain theultrasound-based force sensing device in a relatively dormant state,with the effect of reducing ongoing power use, until such time as thecapacitive touch sensing device indicates that there is a contact ornear contact by the user on the touch I/O device. For a first example,once there is a contact or near contact by the user on the touch I/Odevice, the touch I/O device can activate the ultrasound-based forcesensing device, with the effect that the ultrasound-based force sensingdevice need not draw power at times while the user is not contacting thetouch I/O device. For a second example, once there is a contact or nearcontact by the user on the touch I/O device, the touch I/O device canactivate a portion of the ultrasound-based force sensing deviceassociated with the location where the contact or near contact occurs,with the effect that only those portions of the ultrasound-based forcesensing device need draw power only at locations which are associatedwith places where the user is contacting the touch I/O device.

FIG. 9 is a flow chart that illustrates a method 7000 of processingtouch input in accordance with embodiments discussed herein. The method7000 receives a touch input and measures the force of touch input. In sodoing, the method classifies the force of touch as being light, medium,or heavy.

Initially, in operation 7005, the touch processing module 2026determines if the touch input was entered with greater than a first or“X” amount of force. If the touch input was not entered with greaterthan X amount of force, control may proceed to operation 7010. If thetouch input was entered with a greater than X amount of force, controlmay proceed to operation 7015.

In operation 7010, the touch processing module 2026 executes nooperation because the force of the touch was below a threshold amountthat the system recognizes as a touch. Following operation 7010, controlreturn to the A control point.

In operation 7015, the touch processing module 2026 determines if thetouch input was entered with greater than a second or “Y” amount offorce. If the touch input was not entered with greater than Y amount offorce, control may proceed to operation 7020. If the touch input wasentered with a greater than Y amount of force, control may proceed tooperation 7025.

In operation 7020, the touch processing module 2026 registers a lighttouch. Following operation 7020, the touch processing module 2026 mayexecute a light touch processing sequence B. An example light touchprocessing sequence is described in further detail below in connectionwith FIG. 8.

In operation 7025, the touch processing module 2026 determines if thetouch input was entered with greater than a third or “Z” amount offorce. If the touch input was not entered with greater than Z amount offorce, control may proceed to operation 7030. If the touch input wasentered with a greater than Z amount of force, control may proceed tooperation 7035.

In operation 7030, the touch processing module 2026 registers a mediumtouch. Following operation 7030, the touch processing module 2026 mayexecute a medium touch processing sequence C. An example medium touchprocessing sequence is described in further detail below in connectionwith FIG. 9

In operation 7035, the touch processing module 2026 registers a heavytouch. Following operation 7035, the touch processing module 2026 mayexecute a heavy touch processing sequence D. An example heavy touchprocessing sequence is described in further detail below in connectionwith FIG. 10

FIG. 10 is flow chart that illustrates an example light touch processingsequence in accordance with embodiments discussed herein. The lighttouch processing sequence shown in FIG. 10 is by way of example and notlimitation. It should be appreciated that other light touch processingsequences consistent with this disclosure may be implemented asappropriate in a particular design.

Initially, in operation 8005, the touch processing module 2026determines if the electronic device is currently running an application.If the electronic device is currently running an application, controlmay proceed to operation 8010. If the electronic device is not currentlyrunning an application, control may proceed to operation 8015.

In operation 8010, the touch processing module 2026 exits theapplication and enters the home screen. Following operation 8010,control may return to the A control point.

In operation 8015, the touch processing module 2026 determines if theelectronic device is currently at the home screen. If the electronicdevice is currently at the home screen, control may proceed to operation8020. If the electronic device is not currently at the home screen,control may proceed to operation 8025.

In operation 8020, the touch processing module 2026 exits the homescreen and enters the search function screen. Following operation 8020,control may return to the A control point.

In operation 8025, the touch processing module 2026 determines if theelectronic device is currently at the lock screen. If electronic deviceis currently at the lock screen, control may proceed to operation 8030.If electronic device is not currently at the lock screen, control maycontinue with other steps in light touch processing sequence that areappropriate for modes of operation that the system may implement.

In operation 8030, the touch processing module 2026 executes nooperation because a light touch received when the device is at the lockscreen may merely indicate that an object brushed the surface of thedevice and so no meaningful touch was actually received. Followingoperation 8030, control may return to the A control point.

FIG. 11 is flow chart that illustrates an example medium touchprocessing sequence in accordance with embodiments discussed herein. Themedium touch processing sequence shown in FIG. 11 is by way of exampleand not limitation. It should be appreciated that other medium touchprocessing sequences consistent with this disclosure may be implementedas appropriate in a particular design.

Initially, in operation 9005, the touch processing module 2026determines if the electronic device is currently running an applicationor is at the home screen. If the electronic device is currently runningan application or is at the home screen, control may proceed tooperation 9010. If the electronic device is not currently running anapplication or at the home screen, control may proceed to operation9015.

In operation 9010, the touch processing module 2026 displays the recentapplications selector interface. Following operation 9010, control mayreturn to the A control point.

In operation 9015, the touch processing module 2026 determines if theelectronic device is currently displaying the recent applicationsselector interface. If the electronic device is currently displaying therecent applications selector interface, control may proceed to operation9020. If the electronic device is not currently displaying the recentapplications selector interface, control may proceed to operation 9025.

In operation 9020, the touch processing module 2026 exits the homescreen and enters the search function screen. Following operation 9020,control may return to the A control point.

In operation 9025, the touch processing module 2026 determines if theelectronic device is currently at the lock screen. If electronic deviceis currently at the lock screen, control may proceed to operation 9030.If electronic device is not currently at the lock screen, control maycontinue with other steps in medium touch processing sequence that areappropriate for modes of operation that the system may implement.

In operation 9030, the touch processing module 2026 display controls ofthe music player. Following operation 9030, control may return to the Acontrol point.

FIG. 12 is flow chart that illustrates an example heavy touch processingsequence in accordance with embodiments discussed herein. The heavytouch processing sequence shown in FIG. 12 is by way of example and notlimitation. It should be appreciated that other heavy touch processingsequences consistent with this disclosure may be implemented asappropriate in a particular design.

Initially, in operation 10005, the touch processing module 2026determines if the voice command interface is not currently enabled. Ifthe voice command interface is not currently enabled, control mayproceed to operation 10010. If the voice command interface is currentlyenabled, control may proceed to operation 10015.

In operation 10010, the touch processing module 2026 enables the voicecommand interface. Following operation 10010, control may return to theA control point.

In operation 10015, the touch processing module 2026 executes nooperation. Following operation 10015, control may return to the Acontrol point.

FIG. 13 is a flow chart that illustrates a method 11000 of processingtouch input in accordance with embodiments discussed herein. The method11000 combines command recognition through force-of-touch sensing withother types of command recognition in order to expand the functionalityof the home button.

Initially, in operation 11005, the touch processing module 2026determines the force applied when the user pressed the home button.Following operation 11005, control may proceed to operation 11010.

In operation 11010, the touch processing module 2026 determines theduration during which the user pressed the home button. Followingoperation 11010, control may proceed to operation 11015.

In operation 11015, the touch processing module 2026 sets a timer. Thetimer tracks the time between touch inputs. If the time between inputsis sufficiently short, the system may recognized the two inputs asgrouped together, indicating a particular command that is entered byentering successive touches. Following operation 11015, control mayproceed to operation 11020.

In operation 11020, the touch processing module 2026 determines if theelectronic device has received a further home button input. If theelectronic device has received a further home button input, control mayagain pass to operation 11005. If the electronic device has not receiveda further home button input, control may proceed to operation 11025.

In operation 11025, the touch processing module 2026 determines if thetimer has expired. If the timer has not expired, control may again passto operation 11020. If the timer has expired, control may proceed tooperation 11030.

In operation 11030, the touch processing module 2026 correlates forceand duration information for one or more home-button inputs with anappropriate command. Following operation 11030, the electronic deviceexecutes the command in operation 11035.

Certain aspects of the embodiments described in the present disclosuremay be provided as a computer program product, or software, that mayinclude, for example, a computer-readable storage medium or anon-transitory machine-readable medium having stored thereoninstructions, which may be used to program a computer system (or otherelectronic devices) to perform a process according to the presentdisclosure. A non-transitory machine-readable medium includes anymechanism for storing information in a form (e.g., software, processingapplication) readable by a machine (e.g., a computer). Thenon-transitory machine-readable medium may take the form of, but is notlimited to, a magnetic storage medium (e.g., floppy diskette, videocassette, and so on); optical storage medium (e.g., CD-ROM);magneto-optical storage medium; read only memory (ROM); random accessmemory (RAM); erasable programmable memory (e.g., EPROM and EEPROM);flash memory; and so on.

While the present disclosure has been described with reference tovarious embodiments, it will be understood that these embodiments areillustrative and that the scope of the disclosure is not limited tothem. Many variations, modifications, additions, and improvements arepossible. More generally, embodiments in accordance with the presentdisclosure have been described in the context of particular embodiments.Functionality may be separated or combined in procedures differently invarious embodiments of the disclosure or described with differentterminology. These and other variations, modifications, additions, andimprovements may fall within the scope of the disclosure as defined inthe claims that follow

Timing Diagram

In some embodiments various components of the computing device and/ortouch screen device may be driven or activated separately from eachother and/or on separate frequencies. Separate drive times and/orfrequencies for certain components, such as the display, touch sensor orsensors (if any), and/or force sensors may help to reduce cross-talk andnoise in various components. FIGS. 14A-14C illustrate different timingdiagram examples, each will be discussed in turn below. It should benoted that the timing diagrams discussed herein are meant asillustrative only and many other timing diagrams and driving schemes areenvisioned.

With respect to FIG. 14A, in some embodiments, the display 14 and theforce sensor 18 may be driven substantially simultaneously, with thetouch sensitive component 1001 being driven separately. In other words,the driver circuits for the force sensing device 18 may be activatedduring a time period that the display is also activated. For example,the display signal 30 and the force sensing signal 34 may both be onduring a first time period and then may both inactive as the touchsensing device signal 32 is activated.

With respect to FIG. 14B, in some embodiments, the touch and forcedevices may be driven at substantially the same time and the display maybe driven separately. For example, the display signal 40 may be set high(e.g., active) during a time that the touch signal 42 and the forcesignal 44 may both be low (e.g., inactive), and the display signal 40may be low while both the touch signal 42 and the force signal 44 arehigh. In this example, the touch signal 42 and the force signal 44 mayhave different frequencies. In particular, the touch signal 42 may havea first frequency F1 and the force signal 44 may have a second frequencyF2. By utilizing separate frequencies F1 and F2, the computing devicemay be able to sample both touch inputs and force inputs atsubstantially the same time without one interfering with the other,which in turn may allow the processor to better correlate the touchinputs and the force inputs. In other words, the processor may be ableto correlate a force input to a touch input because the sensors may besampling at substantially the same time as one another. Additionally,the separate frequencies may reduce noise and cross-talk between the twosensors. Although the example in FIG. 14B is discussed with respect tothe force and touch signals, in other embodiments each of the drivesignal, the touch signal, and/or the force signal may have separatefrequencies from each other and may be activated simultaneously orcorrespondingly with another signal.

With respect to FIG. 14C, in some embodiments, various components in thecomputing device may be driven separately from one another. For example,the display signal 50 may be driven high, while both the touch signal 52and the force signal 54 are low. Additionally, the touch signal 52 maybe high while both the force signal 54 and the display signal 50 are lowand similarly the force signal 54 may be high while both the displaysignal 50 and the touch signal 52 are low. In these examples, the forcesignal's active period may be positioned between the active periods ofthe display and the touch sensor. In other words, the force sensor 18may be driven between the display being driven and the touch sensorsbeing driven. In these examples, each of the devices may be active atseparate times from one another, thereby reducing inter-system noise. Insome embodiments, the force sensor may have a shorter drive time thanthe display or touch signals; however, in other embodiments, the forcesensor may have a drive time that is substantially the same as or longerthan the display and/or touch sensor.

Although embodiments have been fully described with reference to theaccompanying drawings, it is to be noted that various changes andmodifications will become apparent to those skilled in the art. Suchchanges and modifications are to be understood as being included withinthe scope of the various embodiments as defined by the appended claims.The foregoing description has broad application. Accordingly, thediscussion of any embodiment is meant only to be an example and is notintended to suggest that the scope of the disclosure, including theclaims, is limited to these examples.

We claim:
 1. A force-sensitive device, comprising: an ultrasound-basedforce-sensing element disposed with respect to a surface of theforce-sensitive device, the force-sensing element comprising: at leastone emitter operative to transmit an energy pulse toward the surface ofthe force-sensitive device; and at least one receiver configured toreceive a reflected energy pulse from the surface of the force-sensitivedevice and output force data corresponding to a touch on the surface ofthe force-sensitive device; and a processor operative to: receive, fromthe force-sensing element, the force data; execute a first process foridentifying an input command when the force data indicates a firstmagnitude of force, the first process comprising: determining whetherthe force-sensitive device is displaying an application or a homescreen; responsive to determining that the force-sensitive device isdisplaying the application, identifying the input command as a commandto display a home screen; and responsive to determining that theforce-sensitive device is displaying the home screen, identifying theinput command as a command to display a search function screen; andexecute a second process for identifying the input command when theforce data indicates a second magnitude of force, wherein the secondprocess is different from the first process.
 2. The force-sensitivedevice of claim 1, wherein: the surface of the force-sensitive device isat least partially defined by a display; and the force-sensitive devicefurther comprises a sensing layer configured to detect a location of thetouch on the display.
 3. The force-sensitive device of claim 1, whereinthe at least one receiver is operatively coupled to the processor andconfigured to transmit a signal to the processor corresponding to thereflected energy pulse.
 4. The force-sensitive device of claim 3,wherein the processor is operative to determine an estimate of a forceapplied to the surface of the force-sensitive device based at least inpart on the reflected energy pulse.
 5. The force-sensitive device ofclaim 1, wherein the second process for identifying the input commandcomprises at least one of: determining whether the force-sensitivedevice is displaying the application or the home screen; determiningwhether the force-sensitive device is displaying a recent applicationsuser interface; or determining whether the force-sensitive device is ina locked state.
 6. The force-sensitive device of claim 5, wherein thesecond process comprises: responsive to determining that theforce-sensitive device is displaying the application or the home screen,identifying the input command as a command to display the recentapplications user interface; responsive to determining that theforce-sensitive device is displaying the recent applications userinterface, identifying the input command as a command to display thehome screen; and responsive to determining that the force-sensitivedevice is in the locked state, identifying the input command as acommand to display music player controls.
 7. The force-sensitive deviceof claim 1, wherein the second process for identifying the input commandcomprises: determining whether a voice command interface is enabled; andresponsive to determining that the voice command interface is notenabled, identifying the command as a command to enable the voicecommand interface.
 8. A method for determining and responding to aninput force, comprising: ultrasonically sensing a force exerted on asurface of an electronic device, by: transmitting an energy pulse towardthe surface of the electronic device; and receiving a reflected energypulse from the surface of the electronic device; determining a magnitudeof the force using the reflected energy pulse; responsive to themagnitude of the force being within a first range of values, executing afirst process for identifying an input command, the first processcomprising: determining whether a voice command interface is enabled;and responsive to determining that the voice command interface is notenabled, determining that the input command is a command to enable thevoice command interface; responsive to the magnitude of the force beingwithin a second range of values different from the first range ofvalues, executing a second process for identifying the input command,wherein the second process is different from the first process; andexecuting the identified input command.
 9. The method of claim 8,wherein: the force exerted on the surface corresponds to a touch at alocation on the surface; the method further comprises detecting thelocation of the touch; and identifying the input command uses both thelocation of the touch and the magnitude of the force.
 10. The method ofclaim 9, wherein: the surface is at least partially defined by a displayof the electronic device; and the location of the touch corresponds to agraphical element rendered by the display.
 11. The method of claim 8,further comprising: determining a duration during which the force isexerted on the surface of the electronic device; and identifying theinput command uses both the duration and the magnitude of the force. 12.An electronic device, comprising: a touch-sensitive display; anultrasound-based force-sensing element positioned below the display, theforce-sensing element comprising: an emitter operative to transmit anultrasonic pulse toward a surface of the display; a receiver configuredto receive a reflected ultrasonic pulse from the surface of the displayand output force data corresponding to a touch on the surface of thedisplay; and a processor operative to: receive the force data andestimate the amount of force; execute a first process for identifying aninput command in response to the amount of force equaling or exceeding afirst threshold, the first process comprising: determining whether theforce-sensitive device is displaying an application or a home screen;responsive to determining that the force-sensitive device is displayingthe application, determining that the input command is a command todisplay a home screen; and responsive to determining that theforce-sensitive device is displaying the home screen, determining thatthe input command is a command to display a search function screen; andexecute a second process for identifying the input command in responseto the amount of force equaling or exceeding a second threshold greaterthan the first threshold, wherein the second process is different fromthe first process.
 13. The electronic device of claim 12, wherein theforce data indicates an amount of force applied to a particular regionof the display.
 14. The electronic device of claim 13, wherein theregion of the display is indicated by an image rendered on the display.15. The electronic device of claim 12, wherein the receiver isoperatively coupled to the processor and configured to transmit a signalto the processor corresponding to the reflected energy pulse.
 16. Theelectronic device of claim 15, wherein the processor is operative todetermine the estimate of force applied to the display based at least inpart on the reflected energy pulse.
 17. The electronic device of claim12, wherein the processor is further configured to: execute a thirdprocess for identifying the input command in response to the amount offorce equaling or exceeding a third threshold greater than the secondthreshold.
 18. The electronic device of claim 12, wherein: the processoris further configured to determine a duration during which the amount offorce is exerted; and at least one of the first process or secondprocess comprises determining the input command using the duration.